ExplorAR is an augmented reality game platform for museums that provides a new experience to learn the world of the past by exploring augmented reality with your phone both inside and outside museums. In this interactive experience, players collect 3D-scanned real-life artifacts such as fossils, sculptures, & paintings. Players will learn how to preserve historical objects by excavating fragments of artifacts from rocks, collaborate by combining fragments of incomplete artifacts, & express their creativity by designing their own virtual gallery.

Museums are places where both adults and children can spend their leisure time. They provide reliable, authentic, and understandable information where the public can find meaning and connection. In this era of digital information where there are more alternatives to leisure activities, museums have a decline in interest as a result from lack of social participation from digital natives. As reported by the New Media Consortium, major museums & institutions start digitizing their collection by 3D scanning and exporting them into 3D models which helps museums expand their audience. There is a need for museums to engage their visitors, increase participatory experience through new media, and create a crowdsourcing environment. Our solution is ExplorAR, a collaborative artifact-based mixed reality exploration game can offer a new learning experience. Unlike other virtual reality museum tours and audio guided museum tours that have been existed, mixed reality gamification is the main aspect of this approach.

ExplorAR integrates both inside & outside museum experience, meaning that it extends the application lifetime. There will be monthly updates of new content and special events for different museums to keep users engaged with the application. With Niantic’s Real World Platform, the application can be used for both Android and iOS devices. And lastly the emulation of meaningful and interactive experiences would make the application less gimmicky. ExplorAR emulates the experience of being an archaeologist or a paleontologist when exploring the world, excavating artifacts, and collaborating with other players and the experience of being a museum curator when decorating their virtual gallery.

Our goal is to create a mixed reality game platform that demonstrates a new gamified learning experience based on exploration which supports public learning and empowers the engagement between users and museums.

We identified 2 targeted main stakeholders of the game: high school & college level students who are motivated to learn subjects outside of their curriculum and museum curators who want to try new platforms to engage and attract more customers.

We interviewed two experienced museum curators who both have spent at least a decade in curating a museum. Based on the interviews, we learned that:

• Even though they come from different departments, they feature some objects in common in their exhibits, such as fossils, artifacts, and art.
• They stated that museums seem to be overshadowed by other modern entertainment media.
• They thought the newer generation is not entirely interested in museums, because they are more exposed to modern entertainment media in their free time.
• They believed that creating a mixed reality game based on museum artifacts would make people, especially the newer generation, experience the sense of exploration that traditional museums could not offer which might also increase more visitors.
• They were currently 3D scanning their artifacts collection as a part of their digitization program, but had no clue how to utilize them.
• They stated a problem where there are some artifacts that have very little information, which would be difficult to be presented in the game because they need to have accurate information.

We also interviewed six students from different department and majors. Based on the interviews, we learned that:

• Although they have different background & interests, they are motivated to learn subjects that are outside of their curriculum.
• Some students love museums because they find it as the best place to learn and interact with interesting subjects.
• They thought the collaborative aspect of the game is very interesting because they can engage with other users who pursue the same learning objective and connect with other users who are playing the game at the same time.
• They thought the newer generations don’t always find museums boring, which depend on their interests.
• They thought that a mixed reality game would be a great to create an engaging learning experience.

Based on the surveys and interviews, we created personas to conceptualize the user’s goals and needs. Personas could help our team see a representation of how the users would benefit by using the game.

Our initial design of our concept was originally for the Microsoft Hololens. The Hololens would be a great platform for the users to experience mixed reality, but we wanted to reduce the complexity of using two devices at once to properly run the game. The Hololens lacks a GPS sensor, which is why we had to pair it with a smartphone via Bluetooth to acquire geospatial data.

We decided to pivot the design into a phone-based augmented reality platform. We settled on using phone based application because it is relatively easier and faster process for people to engage with each other just by downloading the application from the app store. GPS sensor is a prevalent component in smartphones, which we can design the system by using only one device per user. Even though using a smartphone app would offer a completely different experience compared to a head mounted display, we thought this is the best decision for the design to be more scalable for numerous users of the game.

We started designing the mobile application by creating a low-fidelity prototype from a rough sketch. As the concept evolves, the prototype eventually developed into a medium-fidelity prototype created using Photoshop and Marvel.

After the announcement that our project got selected as a top 10 finalist for Niantic's Developer Competition, we started making a high fidelity prototype using Niantic's Real World Platform on Unity.

Map

This is the screenshot of the Main Menu which includes the map. The explorer avatar is located at the center of the map while interactive objects are scattered around the map. The main menu consists of Explore, E-Pedia, Gallery, Settings, & User Profile which will be explained later in this article.

Artifacts

Artifacts are the main collectable items in the game. These artifacts are based on real life 3D scans from museums which include: The Smithsonian Institution, Matson Museum of Anthropology, & Penn State College of Earth & Mineral Science. Some artifacts that are encapsulated in rocks are fragmented and some of them are already completed. Each fragment contains a snippet of information related to the artifact. By combining fragments into a complete artifact, explorers can learn the whole information regarding to the completed artifact in their encyclopedia. Explorers can retrieve fragments of artifacts in pile of rocks that appear in the map. The fragments are generated based on the artifacts of the closest museum to the explorers’ current location. These fragments spawn evenly distributed in a controlled space. Later in the future, different museums will feature their own featured artifacts.

Excavation Mode

Excavation Mode is where explorers extract artifacts from rocks. Tap on the rock to excavate a selected surface of the rock. Tapping the rock increases progress percentage. Once progress percentage has reached 100%, artifact fragment is obtained and goes back to the Map scene. Explorers need to be careful when excavating, if a user taps artifact when progress percentage hasn’t reached 100%, the artifact health bar lowers. If the artifact health bar reaches 0, the excavated artifact is broken and the user goes back to the map.

Combine Mode

In creating an engaging learning experience, explorers can collaborate with each other. By having a common goal to collect and combine artifact fragments, explorers will collaborate with each other where they can combine missing fragments into a complete artifact. To combine fragments into an artifact, explorers can select which fragment in their inventory they want to combine and simply drag and drop the piece into the other fragment in augmented reality. Both explorers acquire the completed artifact which they can see the artifact in the encyclopedia and use the artifact to decorate their gallery. This feature hopes to increase social engagement, learning motivation and healthy competition between others. and its history. If an artifact was completed through collaboration, the completion date and collaborators are shown.

Encyclopedia

Explorers can see all of their completed artifacts on the encyclopedia. Their phone acts as an adventurer’s log book & mini encyclopedia that tracks all of their discovered artifacts. The encyclopedia can be sorted into different museums where explorers can read interesting facts about their findings and observe the 3D model by rotating and zooming the object. After the completion of an artifact, it will show the whole information about the artifact including facts and its history. If an artifact was completed through collaboration, the completion date and collaborators are shown.

Virtual Gallery

Explorers can see all of their completed artifacts on the encyclopedia. Their phone acts as an adventurer’s log book & mini encyclopedia that tracks all of their discovered artifacts. The encyclopedia can be sorted into different museums where explorers can read interesting facts about their findings and observe the 3D model by rotating and zooming the object. After the completion of an artifact, it will show the whole information about the artifact including facts and its history. If an artifact was completed through collaboration, the completion date and collaborators are shown.

Explorers can edit their gallery by selecting the Edit button after they placed their portal on the tracked surface. There are 5 slots in the gallery which explorers can decorate based on their choice. After decorating their gallery, specific gallery visitors will visit their gallery based on the category of the artifacts.

In the decorated virtual gallery, special guests will also visit the gallery. They have different interests and personalities. Intrigue them by decorating your gallery with different kinds of artifacts in your gallery. They are more than just fictional characters, they need to be understood too just like human beings. Explorers can interact with them and learn more about their backstories after they become recent visitors in the Guest Book. Once explorers have accumulated a lot of visits from other explorers and special characters, they can unlock new features for the gallery.

We asked five students to test the application prototype. Before they were tested, they were explained the main idea and the objectives of the game. After user testing, we asked some questions about the main features of the game and suggested the users to give feedback of each feature. From the users’ feedback, we discovered:

• Users found the interactable objects in the environment and the navigation map immersive.
• Users felt more curious about the artifact after they got a fragment of an artifact which showed a hint about the whole artifact.
• Although some users were lost when combining a fragment, they loved the idea of collaborating with each other.
• Users enjoyed the concept of collecting artifacts and motivated to learn more about the history and factual information of the artifacts.
• Users were interested in the virtual gallery where they can create their own gallery and visit another user’s gallery.
• Although the tested users admitted they don’t know much about artifacts, they believed crowdsourcing unknown artifacts through users’ insights can benefit museums’ research.

We have designed a collaborative artifact-based mixed reality game using human centered design methods. Based on the user testing, users have a lot of interest in the game and the goal of the concept is fulfilled, to create a new gamified learning experience based on exploration that supports public learning and empowers the engagement between users and museums. After minor design tweaks in the future, explorAR can be developed into a working game and can be used as a platform for museums as a part of the digitization movement. The research paper got published in Asian HCI Symposium in CHI'18. The project got a runner up prize in Niantic's Beyond Reality Developer Competition.

ExplorAR has finished its first initial steps, although it is still far from being complete for launch. We almost got everything done for the Outdoor Experience and missed some minor features due to time constraints. We built ExplorAR’s Outdoor Experience throughout the contest, but we expect to have the Indoor Experience on launching the game.

Shared Multiplayer

During the competition, we haven’t got the chance to take a deeper dive into the shared experience due to time constraints and other milestone requirements.

Crowdsourcing Mode

Based on the previous user research, museum curators mentioned they need a crowdsourcing platform that allows users to contribute knowledge of a lesser known artifact. Explorers may come from different cultural backgrounds which would be beneficial for the museum if they could provide information about the artifact. Museum Curators are free to provide lesser known artifacts in the game platform for their research. If the museum finds the information from explorers to be valid, they will update the information to the artifact’s encyclopedia entry and the user will receive a reward from the museum.

Indoor Experience

In Indoor Museum Experience, exclusive artifacts can be encountered inside the corresponding museum where explorers answer interactive quizzes to obtain them. Indoor Museum Experience artifacts are exclusive inside the corresponding museum. Explorers need to answer quizzes to obtain these exclusive artifacts in the museum. Clues about the artifact can be found throughout the museum exhibition.

There will be additional experiences in the museum where players can interact with the museum exhibition, such as Time Portals, Glyph Translator, Dino Revival, & Style Transfer Camera Filters. These features will be developed in the future after the outdoor experience is developed.

For a non-profit game, it requires proper funding from educational institutions for future updates and maintenance. We also need to reach out to more museums that could provide 3D scans of artifacts for more variety of collectible content. User feedback from usability tests is still an important aspect of the development of the project as well to test new features before launch.